Videos distributed via TV broadcast and videos stored on recording media, such as video tapes, DVDs, or the like, are often interlaced videos. The interlaced image Fi is an image decimated every other line in the image scanning direction, as represented by the equation (1), different from the progressive image Fp.
                    [                  Equation          ⁢                                          ⁢          1                ]                                                                      Fi          ⁡                      (                          x              ,              y                        )                          =                  {                                                                                                                Fp                      ⁡                                              (                                                  x                          ,                          y                                                )                                                                                                                        if                      ⁢                                                                                          ⁢                                              (                                                                              y                            ⁢                                                                                                                  ⁢                            mod                            ⁢                                                                                                                  ⁢                            2                                                    =                          0                                                )                                                                                                                                  Null                                                        else                                                              ⁢                                                          ⁢              or              ⁢                                                          ⁢              Fi              ⁢                              (                                  x                  ,                  y                                )                                      =                          {                                                                                          Fp                      ⁡                                              (                                                  x                          ,                          y                                                )                                                                                                                        if                      ⁢                                                                                          ⁢                                              (                                                                              y                            ⁢                                                                                                                  ⁢                            mod                            ⁢                                                                                                                  ⁢                            2                                                    =                          1                                                )                                                                                                                                  Null                                                        else                                                                                                          (        1        )            
where Fi(x, y) and Fp(x, y) represent pixel values Fi and Fp in the coordinate (x, y), respectively. X mod y is an arithmetic symbol representing a remainder of x/y.
When an image display unit, such as LCD or plasma display, that manifests progressive images, displays interlaced images or when the image collating unit that compares an input progressive image with a corresponding progressive image stored preliminarily in a database and recognizes the progressive image, receives interlaced images, Null pixels on lines decimated in the interlaced images, represented by the equation (1), have to be restored via interpolation to generate a progressive video. This interpolation process is generally called as the interlacing to progressive conversion (IP conversion, De-interlacing). Hereinafter, Null pixels to be interpolated are called as interpolated pixels.
As one of IP converting methods is cited the method using the peripheral pixel information for the interpolation of the Null pixel on the coordinate (x1, y1) within Fi. As that method, there are the simple linear interpolation method represented by the equation (2) and the edge adaptive interpolation method (non-patent document 1 and patent document 1) represented by the equation (3). However, Fp2 is a progressive video generated through the above interpolation method and Fp2(x,y) is a pixel value Fp2 on the coordinate (x,y). m in the equation (3) corresponds to p minimizing the formula (4) within the preliminarily determined range −Φ≦p≦Φ. Hereinafter, the range (−Φ≦p≦Φ) is called as a search range and the numerical value Φ is called as a search range decision value.
Moreover, referring to the non-patent document 2, the change in luminance of the upper and lower lines within a search range by the edge adaptive interpolation method for respective interpolation pixels are classified into five patterns based on the luminance conversion of peripheral pixels, as shown in FIG. 7. Five patterns are (1) one characteristic being flat, (2) both characteristics increasing and decreasing monotonously in the same direction, (3) both characteristics being curved convexly in the same direction, (4) one characteristic decreasing and increasing and the other being curved convexly, and (5) others. The search range decision value Φ changes adaptively such that the pattern of a luminance change of the upper or lower lines within a search range belongs to any one of the patterns (1) to (4) and such that the maximum range is a search range. Hereinafter, luminance change patterns of upper and lower lines within a classified search range are called as luminance change patterns of upper and lower lines.
                    [                  Equation          ⁢                                          ⁢          2                ]                                                                      Fp          ⁢                                          ⁢          2          ⁢                      (                                          x                ⁢                                                                  ⁢                1                            ,                              y                ⁢                                                                  ⁢                1                                      )                          =                                            Fi              ⁡                              (                                                      x                    ⁢                                                                                  ⁢                    1                                    ,                                                            y                      ⁢                                                                                          ⁢                      1                                        -                    1                                                  )                                      +                          Fi              ⁡                              (                                                      x                    ⁢                                                                                  ⁢                    1                                    ,                                                            y                      ⁢                                                                                          ⁢                      1                                        +                    1                                                  )                                              2                                    (        2        )                                [                  Equation          ⁢                                          ⁢          3                ]                                                                      Fp          ⁢                                          ⁢          2          ⁢                      (                                          x                ⁢                                                                  ⁢                1                            ,                              y                ⁢                                                                  ⁢                1                                      )                          =                                            Fi              ⁡                              (                                                                            x                      ⁢                                                                                          ⁢                      1                                        -                    m                                    ,                                                            y                      ⁢                                                                                          ⁢                      1                                        -                    1                                                  )                                      +                          Fi              ⁡                              (                                                                            x                      ⁢                                                                                          ⁢                      1                                        +                    m                                    ,                                                            y                      ⁢                                                                                          ⁢                      1                                        +                    1                                                  )                                              2                                    (        3        )                                [                  Equation          ⁢                                          ⁢          4                ]                                                                      Sub          ⁡                      (            p            )                          =                                                      Fi              ⁡                              (                                                                            x                      ⁢                                                                                          ⁢                      1                                        -                    p                                    ,                                                            y                      ⁢                                                                                          ⁢                      1                                        -                    1                                                  )                                      -                          Fi              ⁡                              (                                                                            x                      ⁢                                                                                          ⁢                      1                                        +                    p                                    ,                                      y                    +                    1                                                  )                                                                                  (        4        )            
As for IP conversion, it is generally known that interpolating all Null pixels correctly is difficult. That is, that remark means that the progressive image generated via the IP conversion contains pixels interpolated erroneously (hereinafter, interpolated pixels).
As to image processing application apparatuses, such as video display units or image collating units, that utilize progressive images mentioned above, the problem is that the processing performance due to erroneously interpolated pixels contained in a progressive image is degraded. In order to prevent such problem, the above-mentioned technique includes the steps of calculating the interpolation reliability for each interpolated pixel, based on difference absolute values of a pixel utilized for interpolation, and performing image processing according to the interpolation reliability in the image processing application. Hereinafter, the difference absolute value of a pixel used for interpolation is called as an interpolated difference value.
For example, the patent document 2 discloses the image processing application apparatus that performs the motion adaptive IP conversion, which synthesizes a progressive image interpolated from one interlaced image at a current time and an interlaced image at other time according to the previously described method, using still/motion discrimination, to display progressive images interpolated at higher precision. In such image processing application apparatus, the process changing is executed with the interpolation reliability, as one factor, calculated based on the interpolation difference value. The calculation is carried out in such way that the interpolation reliability has a larger value when the interpolation difference value is small and has a smaller value when the interpolation difference value is large.
Referring to FIG. 9, the image processing apparatus will be explained below that generates progressive images through IP conversion in the technology related to the present invention and outputs them and their interpolation reliabilities. FIG. 9 is a block diagram illustrating the configuration of an image processing apparatus that generates progressive images in the technology related to the present invention and calculates interpolation reliabilities thereof.
Referring to FIG. 9, the image processing apparatus 100 includes an image interpolation means 11 and an interpolation reliability calculation means 12. The image processing apparatus 100 receives interlaced images and outputs progressive images and their interpolation reliabilities to an image processing application apparatus 400, such as video display unit or image collating unit.
The image interpolation means 11 receives an interlaced image and interpolates it via the method, previously described, and generates a progressive image while outputting an interpolation difference value for each pixel.
The interpolation reliability calculation means 12 receives an interpolation difference value for each pixel output from the image interpolation means 11 and calculates and outputs the interpolation reliability in accordance with the interpolation difference value.
Patent document 1: Japanese patent Laid-open publication No. Hei4-355581
Patent document 2: Japanese patent Laid-open publication No. 2000-50212
Non-patent document 1: “Deinterlacing-an overview”, De Haan, G, Betters, E. B, Proceedings of the IEEE, Volume 86, Issue 9, September 1998 page(s): 1839-1857
Non-patent document 2: “Edge adaptive interlace to progressive conversion method based on a change in luminance around peripheral area”, Toda et al., Sixth information Science Technology Forum, 1-034, 2007